Our model consists of a perceptual front-end based on convolutional neural networks and a dynamics predictor based on interaction networks.

Neural Information Processing Systems http://nips.cc/

MagNet: Discovering Multi-agent Interaction Dynamics using Neural Network Priyabrata Saha, Arslan Ali, Burhan A. Mudassar, Y un Long and Saibal Mukhopadhyay Abstract -- We present the MagNet, a multi-agent interaction network to discover governing dynamics and predict evolution of a complex system from observations. We formulate a multi-agent system as a coupled nonlinear network with a generic ordinary differential equation (ODE) based state evolution, and develop a neural network based realization of its time-discretized model. MagNet is trained to discover the core dynamics of a multi-agent system from observations, and tuned online to learn agent-specific parameters of the dynamics to ensure accurate prediction even when physical or relational attributes of agents, or number of agents change. We evaluate MagNet on point-mass system in two-dimensional space, Ku-ramoto phase synchronization dynamics and predator-swarm interaction dynamics demonstrating orders of magnitude improvement in prediction accuracy over traditional deep learning models. I NTRODUCTION Multi-agent systems are prevalent in both the natural world and engineered world. Engineered distributed systems of mobile robots, multiple sensors, unmanned aerial vehicles etc. often take inspiration from natural multi-agent systems like swarms, schools, flocks, and herds of social animals or birds. Understanding the behavior of such natural or engineered multi-agent systems from sensory observations is a key challenge in robotics from the design and adversarial perspective. Discovering the hidden dynamics of a multi-agent interaction from observations will enable machines to simulate and predict evolution of complex systems.

From just a glance, humans can make rich predictions about the future of a wide range of physical systems. On the other hand, modern approaches from engineering, robotics, and graphics are often restricted to narrow domains or require information about the underlying state. We introduce the Visual Interaction Network, a general-purpose model for learning the dynamics of a physical system from raw visual observations. Our model consists of a perceptual front-end based on convolutional neural networks and a dynamics predictor based on interaction networks. Through joint training, the perceptual front-end learns to parse a dynamic visual scene into a set of factored latent object representations. The dynamics predictor learns to roll these states forward in time by computing their interactions, producing a predicted physical trajectory of arbitrary length. We found that from just six input video frames the Visual Interaction Network can generate accurate future trajectories of hundreds of time steps on a wide range of physical systems. Our model can also be applied to scenes with invisible objects, inferring their future states from their effects on the visible objects, and can implicitly infer the unknown mass of objects. This work opens new opportunities for model-based decision-making and planning from raw sensory observations in complex physical environments.

Last year, for the first time, an artificial intelligence called AlphaGo beat the ranking human champion in a game of Go. This victory was both unprecedented and unexpected, given the immense complexity of the Chinese board game. While AlphaGo's victory was certainly impressive, this artificial intelligence, which has since beat a number of other Go champions, is still considered "narrow" AI--that is, a type of artificial intelligence that can only outperform a human in a very limited domain of tasks. So even though it might be able to kick your ass at one of the most complicated board games in existence, you wouldn't exactly want to depend on AlphaGo for even the most mundane daily tasks, like making you a cup of tea or scheduling a tuneup for your car. In contrast, the AI often depicted in science fiction is called "general" artificial intelligence, which means that it has the same level and diversity of intelligence as a human.